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Abstract Body 


Background / Context: 

In recent years, the early childhood field and the systems that support and govern it have placed 
an increased emphasis on data collection and data use for continuous quality improvement, 
professional development, and compliance and accountability. Indeed, the proliferation of 
Quality Rating and Improvement Systems in states across the country and the Head Start Act 
reauthorization in 2007 have pushed early childhood programs to embrace a more intensive 
culture of “data-driven decision-making” (OHS, 2013). In addition, the newly proposed Head 
Start Perfonnance Standards would require programs to “implement a process for using data to 
identify program strengths and needs, develop and implement plans that address program needs, 
and continually evaluate progress towards achieving program performance goals” (OHS, 2015). 
But despite this growing expectation of data use, there is as yet only an emerging body of 
evidence on effective models of data use in early childhood education settings, and even less 
information about how to measure its frequency and intensity. The Educare Learning Network’s 
Implementation Study (IS) provides an opportunity to explore the measurement of data use 
practices in a nationwide sample of high-quality early education programs serving income- 
eligible and at-risk children age birth through five and their families. 

Educare is a network of enhanced Early Head Start (EHS)/Head Start (HS) programs that 
implement innovative Research-Program Partnerships (RPPs) to engage researchers, program 
leaders, staff, and at times, other stakeholders in a collaborative approach to supporting data use 
practices for decision-making and continuous quality improvement at the school and network 
levels. These RPPs have been infonned by the local evaluation partnerships developed for the 
Early Head Start Impact Study (Love et ah, 2005) and the principles of utilization-focused 
evaluation (Patton, 2008) and participatory evaluation (Ryan & DeStefano, 2000). Based on 
research noting the importance of providing data “with sufficient frequency and in contexts that 
will inspire educators to develop, utilize, and disseminate effective interventions” (Downs & 
Strand, 2006), the RPPs use data from a variety of quantitative and qualitative sources (e.g., 
standardized measures to compare children’s skills to national norms, curriculum-based 
assessment, progress monitoring tools, teacher/parent ratings, interviews, focus groups, etc.) to 
provide a fuller picture of program functioning and impact (Scott-Little et ah, 2003). As such, 
the Educare Learning Network offers an ideal context in which to examine the measurement of 
data use. Furthermore, this study represents a critical step in advancing RPP models and the early 
education field’s conceptualization by leveraging the IS’s rich data use data to develop a measure 
of early educators’ data use beliefs and practices. 

Purpose / Objective / Research Question / Focus of Study: 

This study develops a measurement strategy for quantifying data use in center-based early 
education settings. To do so, we seek to identify specific constructs of data use among early 
childhood educators. Specifically, this study aims to (1) identify distinct constructs of data use 
practices and beliefs from a self-report measure of early care and education staff; (2) understand 
how these data use constructs operate at the individual and site levels; and (3) examine 
associations between data use constructs and key early educator individual characteristics and 
experiences of their work environment. 
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Setting: 

Data for this study were collected in 16 Educare sites, located in urban, rural and suburban areas 
across 13 states in the spring of the 2014-2015 school year. Each program is housed in a new or 
remodeled early education center that includes infant, toddler, and preschool classrooms. 

Population / Participants / Subjects: 

The data for this study were collected as part of the Educare Implementation Study and includes 
503 teachers, specifically lead teachers (n=198), assistant teachers (n=172), teacher aids (n=103), 
and other teaching staff (n=28) across 16 Educare sites during the 2014-2015 academic year. The 
sample is racially diverse (44% white, 38% black, and 18% Hispanic). On average, respondents 
have worked in early education for nearly 12 years and at Educare for about 3 years. Nearly all 
respondents were female (97%), and 51% have a B.A. degree or higher. 

The final presentation will also include survey data from 38 master teachers (staff that supervise 
and provide embedded professional development and coaching to teaching teams) and 76 family 
support staff (staff that work directly with families) from these same Educare sites. 

Intervention / Program / Practice: 

Educare is an enhanced, high-quality birth to age 5 early education program designed to reduce 
the achievement gap for children from low-income families (Guss, Norris, Horm, Monroe, & 
Wolfe, 2013; Stein, Freel, Hanson, Pacchiano, & Eiland-Williford, 2013;Yazejian, Bryant, & 
Kennel, 2013). Educare incorporates four innovative practices through partnerships with 
university researchers and community agencies (e.g., public schools, philanthropists) that show 
promise for improving children’s outcomes: data utilization, coaching and ongoing professional 
development, high-quality teaching and interactions, and strong school-family partnerships. 
Although it is an EHS/HS program, Educare represents somewhat of a hybrid between the 
national public program and a model demonstration. Approximately 80% of the funding for 
Educare comes from public sources (HS, but also, depending on geographic location, child care 
subsidies and public pre-kindergarten funds), with the remaining 20% of costs covered by private 
philanthropy. 

Educare has a variety of quality features that go beyond EHS/HS standards, but that draw from 
the literature on effective practices in early care and education. Most salient to this study is that 
collecting and using data for program improvement is a core feature of Educare. All sites have a 
PhD-level local evaluation partner (LEP) who collects child, family, program practice and staff 
survey data that are shared back with the program leaders and staff as part of the RPP processes 
described above and submits the data to a coordinating center for the cross-site study. 

Teachers included in the current study are supervised by master teachers, all with a B.A. and 
76% with an M.A. degree, who each serve 4.5 classrooms, on average. Master teachers provide 
teachers with ongoing professional development and coaching on research-based best practices. 
At least three times a year, teachers use a curriculum-based progress monitoring tool to track 
children’s development, modify goals if necessary, and share results with parents. Annual 
classroom observations and assessments are conducted by trained and reliable LEPs and shared 
with teaching teams for continuous quality improvement. 
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Research Design: 

The current study employs factor analytic techniques to identify latent constructs in early 
educators’ self-report of data use practices and understandings, and explores the presence of 
these constructs at the site-level. As that there is little prior research that attempts to measure 
data use among early educators, the initial analyses included in this abstract are exploratory in 
approach. 

Data Collection and Analysis: 

Data Collection 

Educator surveys were sent to teachers, master teachers, and family support staff in all 16 
Educare schools via email and completed online in the Spring of the 2014-2015 school year. In 
terms of data use, the staff surveys included four items about the amount of data used (scores 
range from 1, “Not enough” to 3, “Too much”); how informed staff felt about different types of 
data (scores ranged from 1, “I need more information” to 3 “I know a lot about these data”); and 
how frequently they participated in data dialogues (scores ranged from 0, “Never” to 4, 
“Monthly”) and used data for various purposes (scores ranged from 1, “Weekly” to 7, “Never”). 
These last scores were reverse coded such that higher scores indicate higher rates of data use, 
understanding, and frequency across all items. The staff survey include questions about more 
general characteristics such as gender, age, race/ethnicity, primary language spoken, education 
level and type, current enrollment in training, and professional experience in the field and 
specifically at Educare. 

Staff surveys also included existing measures of teacher beliefs including a shortened version of 
the Child Care Worker Job Stress Inventory (CCW-JSI), a self-report measure designed to 
capture three dimensions of job stressors experienced by child care providers: job control, job 
resources, and job demands (Curbow et al., 2000). Higher scores on these subscales indicate that 
staff felt more control in their daily activities and routines, had greater resources at their work, 
and experienced more work-related demands. Finally, staff responded to the Modernity Scale, a 
30-item measure of educators’ Traditional, authoritarian beliefs and Progressive, democratic 
beliefs about how young children leam (Schaefer & Edgerton, 1985). 

Analysis 

Exploratory factor analysis was conducted using Geomin oblique rotation to account for 
predicted correlation among data use factors (Browne, 2001) and maximum likelihood 
estimation to account for missing data. Items’ factor membership was defined such that a factor 
included all items with standardized factor loadings above .30 on that factor and lower 
standardized loadings on all other factors. Items with standardized factor loadings below .30 on 
all factors were not retained in any factor. Factor scores were then created by converting all item 
scores to a 0-1 scale and averaging items within each factor. We examined the associations of 
our newly identified construct factors with individual teacher characteristics to understand how 
self-reported data use is related to key teacher qualifications (e.g., education, experience) and 
beliefs (e.g., job stress, modernity). Finally, we examined the intraclass correlations (ICCs) of 
factor scores as a first step towards understanding whether teacher self-report can help us to 
understand site-level constructs of data use. Future analyses to be included in the final 
presentation will enhance our understanding of site-level data use by repeating these processes 
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with data collected from master teachers and family support staff, and conducting confirmatory 
and sensitivity analyses to compare findings across staff roles, sites, and model specifications. 

Findings / Results: 

Exploratory factor analysis results indicate two underlying data use factors (% =2005.24, df= 229, 
/K.001). Four items related to the amount of data collected were dropped due to standardized 
factor loadings below .30 on both factors. The Data Use Practices factor includes 14 items 
related to the frequency with which early educators use data in a variety of ways and the 
Informed about Data factor includes 6 items related to early educators’ familiarity with different 
types of data (see Table 1). Both factors demonstrate high internal consistency (a=.91 for Data 
Use Practices and a=.85 for Informed About Data). Scores on each range between 0 and 1, with 
means of .39 and .49 and standard deviations of .22 and .24, respectively. The final factors are 
moderately correlated with one another (r=.39, /;<.() 1 ) and with key teacher characteristics and 
qualifications in the expected directions (see Table 2). Finally, our analyses provide preliminary 
evidence that both Data Use Practices and Informed About Data may also be salient constructs at 
the site level (ICC=.52 for both factors); future analyses to be included in the final presentation 
will further explore site-level measurement of data use. 

Conclusions: 

Two latent data use constructs were identified from the analysis of teachers’ self-report measures 
of data use: Infonned About Data and Data Use Practices. These data use constructs are 
positively correlated with one another, suggesting that feeling informed about data and using 
data do not often occur independently. In addition, preliminary analyses suggest that each 
construct is related to key individual educator characteristics in ways that we might expect, 
providing initial evidence of the concurrent validity of these new constructs. Of particular 
interest is that both constructs were positively correlated with experiencing more/better reflective 
supervision and interdisciplinary practice among teaching and family support staff. Additional 
analyses to be included in the final presentation will explore these area further, with the goal of 
understanding data use among master teachers and family support staff — as well as teachers. 
Finally, preliminary analyses suggest that 52% of the variance in each construct lies between 
sites rather than between individuals within sites; additional analyses will explore this notable 
finding further in an effort to define data use constructs at the site-level as well as the individual- 
level. 

Given the current emphasis on data use in the field of early education and social policy more 
broadly, there is growing need to measure data use practices, knowledge, and experience among 
early educators. Such measures could help to inform program improvement, professional 
development, and program monitoring. This study represents a first step towards identifying 
robust measures of data use in center-based early childhood programs across the country. 
However, the data for the current study come from a network of high-quality early childhood 
programs with unusually robust RPP supports in place. As such, future research is needed to 
explore the associations of these new measures with indicators of program quality and child 
outcomes, as well as the infrastructure, processes, and resources necessary to support strong data 
use practices in the diversity of early education programs across the country. 
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Appendix B. Tables and Figures 


Table 1. Exploratory factor analysis factor loadings. 



Data Use 
Practices 

Infonned 

About 

Data 

Shape site's overall data use plan 

.42 

.01 

Data use with LEP 

.31 

.07 

Use data to set professional development goals 

.41 

.01 

Use data to monitor child progress 

.34 

.10 

Use data to with families 

.44 

.07 

Staff dialogues about individual parent interviews 

.71 

-.02 

Staff dialogues about individual child assessments 

.55 

.11 

Staff dialogues about classroom-level child/family data 

.73 

-.04 

Staff dialogues about site-level child/family data 

.83 

-.06 

Staff dialogues about school's community data 

.85 

-.07 

Family dialogues about individual parent interviews 

.79 

-.01 

Family dialogues about individual child assessments 

.58 

.06 

Family dialogues about site-level child/family data 

.87 

-.11 

Family dialogues about school's community data 

.88 

-.09 

Infonned about child assessment data 

.04 

.69 

Informed about parent interview data 

.07 

.70 

Infonned about classroom observation data 

.12 

.72 

Informed about staff survey data 

.07 

.75 

Infonned about school community data 

.18 

.59 

Informed about school quality data 

.14 

.61 

Amount of child data 

.00 

.13 

Amount of family data 

.04 

.13 

Amount of school data 

.01 

.12 

Amount of data reports 

.02 

.14 


Note: Standardized factor loadings >3 are bolded. 
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Table 2. Correlations of factor scores with key teacher characteristics 



1 

2 

1 . Data Use Practices Factor Score 

2. Informed about Data Factor Score 

39 ** 


3. Years taught at Educare 

.06 

14** 

4. Years worked in early care and education 

.04 

.09* 

5. Job Control 

.26** 

. 12 ** 

6 . Job Rewards 

. 22 ** 

.18** 

7. Job Stress 

-. 12 ** 

_ 14 ** 

8 . Modernity Scale 

-.03 

.09 

9. Work Environment 

23** 

19** 

10. Reflective supervision scale 

.26** 

.31** 

11. Teacher- family support interaction 

.38** 

.25** 


* p<. 05; **p<.01 
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